This Page Is Inserted by IFW Operations 
and is not a part of the Official Record 



BEST AVAILABLE IMAGES 



Defective images within this document are accurate representations of 
the original documents submitted by the applicant. 

Defects in the images may include (but are not limited to): 



BLACK BORDERS 

TEXT CUT OFF AT TOP, BOTTOM OR SIDES 
FADED TEXT 
ILLEGIBLE TEXT 
SKEWED/SLANTED IMAGES 
COLORED PHOTOS 

BLACK OR VERY BLACK AND WHITE DARK PHOTOS 
GRAY SCALE DOCUMENTS 



IMAGES ARE BEST AVAILABLE COPY. 

As rescanning documents will not correct images, 
please do not report the images to the 
Image Problem Mailbox. 



(19) 



J 



(12) 



(43) Date of publication: 

18.10.2000 Bulletin 2000/42 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets (11) EP 1 045 578 A2 

EUROPEAN PATENT APPLICATION 

(51) Int CL 7 : H04N 5/217 



(21) Application number: 00302082.3 

(22) Date of filing: 15.03.2000 



(84) 


Designated Contracting States: 


(72) 


Inventors: 




AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU 


• 


Chen, Zhiliang Julian, 




MCNLPTSE 




Piano, Texas 75093, (US) 




Designated Extension States: 


• 


Dierschke, Eugene G., 




AL LT LV MK RO SI 




Dallas, Texas 75248, (US) 






• 


Clynes, Steven Derek, 


(30) 


Priority: 15.03.1999 US 124408 P 




Allen, Texas 75002 (US) 




30.12.1999 US 475652 


• 


Liu, Anil, 




15.03.1999 US 124538 P 




Piano, Texas 75024, (US) 


(71) 


Applicant: 


(74) 


Representative: 




Texas Instruments Incorporated 




Legg, Cyrus James Grahame etal 




Dallas, Texas 75251 (US) 




ABEL & IMRAY, 








20 Red Lion Street 








London WC1R4PQ (GB) 



CM 
< 
00 

LO 
LO 

o 



(54) Filtering of defective picture elements in digital imagers 



(57) Defective pixels in a CMOS array give rise to 
spot noise that diminishes the integrity of the resulting 
image. Because CMOS arrays and digital logic can be 
fabricated on the same integrated circuit using the same 
processing technology and relatively inexpensive and 
fast circuit can be employed to digitally filter the pixel 
data stream and to identify pixels having values that do 
not fall in the range defined by the immediately neigh- 
boring pixels and the deviate from the neighboring pix- 



els by more than a threshold amount. Such conditions 
would indicate that the deviation is caused by a defec- 
tive pixel rather than by desired image data. The thresh- 
old amount can be preprogrammed or can be provided 
by a user or can be dynamically set using feedback indi- 
cating image quality. The filter would also provide a 
solution for other sensors such as CCD. 
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Description 

[0001 ] This application claims benefit of U.S. Patent 
Applications Nos. 60/124408 and 60/124538 filed on 15 
March 1 999 and 09/475652 filed on 30 December 1 999 5 
which applications are incorporated herein by refer- 
ence. Copies of US 60/124408 and US 60/124538 are 
available for inspection in the file of this application. 
[0002] This application is related to and accompa- 
nied by a copy of a commonly assigned, co-pending 
European patent application entitled "Digital imaging 
control with selective intensity resolution enhancement" 
(Applicant Docket No. TI-29033) filed concurrently here- 
with and incorporated herein by reference. A copy of the 
co-pending European patent application entitled "Digital 
imaging control with selective intensity enhancement" is 
available for inspection in the file of this application. 
[0003] This invention relates generally to digital 
imaging devices and specifically to techniques and cir- 
cuits for the filtering of defective picture elements. The 
term pixels is used below to mean picture elements. 
[0004] Digital imaging devices are becoming 
increasingly popular in a variety of applications, includ- 
ing digital cameras, fingerprint recognition, digital scan- 
ners and copiers, and the like. Typical prior art digital 
imaging devices are based on charge coupled device 
(CCD) technology. CCD devices have an array of CCD 
cells, each cell comprising a pixel. Each CCD pixel out- 
puts a voltage signal in proportion to the intensity of light 
impinging upon the cell. This analog voltage signal can 
be converted to a digital signal for further processing, 
digital filtering, storage and the like. As is well known in 
the art, a two dimensional digital image can be con- 
structed from the voltage signals output from a two- 
dimensional array of CCD cells, commonly referred to 
as a sensor array. 

[0005] CCD arrays have a shortcoming in that CCD 
fabrication requires a special process that is not com- 
patible with standard CMOS processes. Thus, the CCD 
array cannot be easily integrated with other logic cir- 
cuits, such as CCD control logic, analog to digital con- 
verters and the like. Additionally, in operation, a CCD 
array requires multiple high voltage supplies from 5V to 
1 2V and CCD arrays tend to consume a large amount of 
power in use. 

[0006] An alternative to CCD arrays is using an 
array formed of CMOS cells. A CMOS sensor array can 
be fabricated using standard CMOS processing and 
thus can be integrated onto a single chip with other cir- 
cuits, such as array control logic, analog to digital con- 
verters (A/D's), digital signal processing (DSP) cores 
and the like. CMOS arrays provide the additional advan- 
tage of operating with a single low supply voltage such 
as 3.3V or 5V, and consuming less power than a com- 
parable CCD array. Finally, a CMOS array can be fabri- 
cated at a lower cost than a similar CCD array. 
[0007] One common problem with both CCD and 
with CMOS imaging devices is that of point defects 



which cause "spot noise" on the image, such as white 
spots on a dark portion of the image or a dark spot on a 
white portion of the image. In CMOS imaging devices, 
white spots are due to pixels (i.e. CMOS cells) with 
excessive leakage current. Dark spots are due to either 
particles covering the pixel or a defect in the pixel elec- 
tronics causing the pixel not to turn on. Spot noise seri- 
ously limits the yield of CMOS imaging devices, 
resulting in increased costs. 

[0008] One method to remove spot noise electroni- 
cally has been proposed by Younse et al. in U.S. Patent 
4,805,023. The Younse et al. implementation requires 
expensive EPROM memory and involves a complicated 
hardware system, further increasing the cost of the 
imaging device. Furthermore, the solution proposed by 
Younse et al. cannot remove temperature dependent 
spot noise, such as white spots appearing only at high 
temperatures. 

[0009] Therefore, a need exists for a relatively inex- 
pensive filter that can quickly and reliably filter out the 
effects of defective pixels, including spot noise, from an 
image signal. 

[0010] The invention provides a filter for filtering an 
image which includes a plurality of picture elements 
each represented by a luminance or a chrominance 
value, including: 

means for selecting a picture element and its two 
adjacent picture elements, 
means for determining whether the selected picture 
element lies between the two adjacent picture ele- 
ments in value, 

means for determining whether the selected picture 
element differs in value from both of the two adja- 
cent picture elements by more than a set amount 
and 

means for replacing the selected picture element, in 
value, by a value dependent on the two adjacent 
picture elements, when, in value, the selected pic- 
ture element does not lie between the two adjacent 
picture elements and differs from both of the two 
adjacent picture elements by more than the set 
amount and 

means for repeating the procedure for further pic- 
ture elements serving as the selected picture ele- 
ment. 

[0011] One embodiment of the invention is a digital 
imaging device comprising a substrate, a sensor array 
formed on the substrate, the array generating an electri- 
cal signal corresponding to the amount of light imping- 
ing upon the array, and imaging logic formed on the 
substrate, coupled to the sensor array and receiving the 
electrical signal. The imaging logic includes an analog 
to digital converter receiving the electrical signal and 
outputting digital pixel values and a defective pixel filter 
receiving the digital pixel values and detecting defective 
pixels on the basis of variations between a selected 
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pixel value and its neighboring pixel values. 
[0012] Another embodiment of the invention is a 
digital imager comprising a lens mechanism, a sensor 
array positioned within a focal plane of said lens mech- 
anism, and an analog buffer and amplifier coupled to an 5 
output of said sensor array, and imaging logic coupled 
to said amplifier. The imaging logic includes a defective 
pixel filter comprising means for detecting whether a 
first pixel is outside an acceptable range defined by 
luminance values of first and second neighboring pixels, 10 
means for determining whether said first pixel deviates 
from said first neighboring pixel by greater than a 
threshold value and means for determining whether 
said first pixel deviates from said second neighboring 
pixel by greater than a threshold value, means for calcu- 15 
lating a corrected pixel value, and means for substitut- 
ing said corrected pixel value for said first pixel if said 
first pixel is outside said acceptable range and said first 
pixel deviates from said first neighboring pixel by greater 
than a threshold value and said first pixel deviates from 20 
said second neighboring pixel by greater than a thresh- 
old value. 

[0013] The invention also provides a method of fil- 
tering an image including a plurality of picture elements 
each represented by a luminance or a chrominance 25 
value, including the steps of: 

selecting a picture element and its two adjacent pic- 
ture elements, 

determining whether the selected picture element 30 
lies, in value, between the two adjacent picture ele- 
ments, 

determining whether the selected picture element 
differs, in value, from both of the two adjacent pic- 
ture elements by more than a set amount and 35 
when the selected picture element does not lie, in 
value, between the two adjacent picture elements 
and differs from both of the two adjacent picture 
elements by more than the set amount, 
replacing the selected picture element, in value, by 40 
a value dependent on the two adjacent picture ele- 
ments, 

the procedure being repeated for further picture 
elements serving as the selected picture element. 

45 

[0014] A method is provided for detecting a defec- 
tive pixel based upon the luminance values generated 
by the pixel and its two nearest neighbors. The method 
includes determining a first difference value between 
the luminance value of the pixel and the luminance so 
value of a first neighboring pixel and comparing the first 
difference value to a pre-determined threshold value. 
The method also includes determining a second differ- 
ence value between the luminance of the pixel and the 
luminance value of a second neighboring pixel and 55 
comparing the second difference value to the pre-deter- 
mined threshold value. Finally, the method includes 
detecting whether the luminance value for the pixel falls 



within an acceptable range defined by the luminance 
value for the first neighboring pixel and the luminance 
value for the second neighboring pixel and identifying 
the pixel as defective if the luminance value for the pixel 
does not fall within the acceptable range and neither the 
first difference value nor the second difference value is 
less than or equal to the threshold value. 
[0015] The above features of the present invention 
will be more clearly understood from consideration of 
the following descriptions in connection with accompa- 
nying drawings in which: 

Figure 1 illustrates a digital imaging device in which 
preferred embodiments of the invention may be 
employed; 

Figure 2 is a block diagram of a preferred embodi- 
ment single chip CMOS imaging device; 
Figures 3a through 3j illustrate various acceptable 
and unacceptable variations between neighboring 
pixels and 

Figures 4a through 4c illustrate the preferred 

embodiment filter of defective pixels operating as a 

moving window of analysis; 

Figure 5 schematically illustrates a first preferred 

embodiment circuit for filtering out defective pixels 

and 

Figure 6 illustrates a portion of a color sensor array. 

[0016] The making and use of the various embodi- 
ments are discussed below in detail. However, it should 
be appreciated that the present invention provides 
many applicable inventive concepts which can be 
embodied in a wide variety of specific contexts. The 
specific embodiments discussed are merely illustrative 
of specific ways to make and use the invention, and do 
not limit the scope of the invention. 
[0017] Figure 1 illustrates a digital imaging device 2 
utilizing preferred embodiments of the present inven- 
tion. Digital imaging device 2 includes a lens mecha- 
nism 4 which receives light emanating or reflected from 
an object to be imaged and focuses the incoming light 
upon an array sensor 6. The array sensor 6 is preferably 
a CMOS sensor array of the type fully described in co- 
pending patent application 09/223,166, entitled Fast 
Frame Readout Architecture for Array Sensors with 
Integrated Correlated Double Sampling System, which 
application is incorporated herein by reference. As will 
be known to one skilled in the art, the sensor array 6 is 
comprised of a two dimensional array of CMOS sen- 
sors, each sensor corresponding to a pixel of the result- 
ing image. Each sensor outputs an analog voltage 
signal, which signal is in proportion to the intensity of 
light impinging upon the particular sensor. The voltage 
signal from each sensor can be scanned in a raster for- 
mat as is well known in the art to generate an analog 
image signal. This analog image signal is fed to imaging 
logic 8 where the analog signal is buffered and amplified 
before being converted to a digital signal. The digital 



5 



EP 1 045 578 A2 



6 



signal can be filtered or further processed before being 
passed to an input / output (IO) port 10 in the form of 
pixel intensity data. Alternatively, with additional signal 
processing, the digital signal can be output in the form 
of a bitmap or other well known digital picture format. 
[0018] Alternatively, the digital signal can be 
passed to a memory 12 for storage. The memory 12 is 
preferably a dynamic random access memory or a static 
random access memory. Alternatively, the memory 12 
could be a magnetic or optical storage device, such as 
a magnetic diskette device, a CD-ROM, or other storage 
device. In each case, an appropriate device controller 
and interface (not shown) would be included along with 
the memory 12. The imaging logic 8, memory 12, and 
I/O port 10 are preferably under the control of a micro- 
processor 14, which is preferably a general purpose 
digital microprocessor operating under instructions 
stored in the memory 12 or a ROM 16. Stored instruc- 
tions could also be provided via the I/O port 10 directly 
to the microprocessor 14 or stored in the memory 12 or 
the ROM 16. 

[0019] In the preferred embodiment, the sensor 
array 6 is formed of an array of CMOS sensor cells, thus 
allowing for the imaging logic 8 to be formed using 
CMOS processes on a single integrated circuit along 
with the sensor array 6. Figure 2 illustrates a single inte- 
grated circuit (IC) 20, upon which is realized both the 
sensor array 6 and the imaging logic 8 according to the 
preferred embodiment of the invention. Other features 
and circuits may be included within the IC 20 including 
internal control registers, microprocessor interface 
logic, memory interface logic and the like. These fea- 
tures have not been illustrated as they are not neces- 
sary for an understanding of the present invention. 
[0020] Further details of the imaging logic 8 will now 
be described with reference to Figure 2. The main path 
for imaging signals is indicated by heavy arrows. The 
analog signal from the sensor array 6 is passed to the 
buffer 22 where the signal is buffered to strengthen the 
signal and fixed pattern noise is removed. From the 
buffer 22 the buffered analog signal is sent to a first 
input of a programmable gain differential amplifier 24. 
The second input of the amplifier 24 receives a refer- 
ence voltage V RER which is fed from a reference control 
block 26 under the control of a digital signal controller 
28. The amplifier 24 also receives a gain control signal 
from the gain control block 30, which operates underthe 
control of a digital signal controller 28. 
[0021] The amplifier 24 maps the two inputs to fully 
differential outputs 25 and 27. In other words, the out- 
puts 25 and 27 correspond to the difference between 
the value of the two input signals (i.e. the analog image 
signal and the reference voltage Vr^f) multiplied by the 
gain value and centered about a common mode voltage 
level. These fully differential outputs 25 and 27 are then 
fed to the inputs of a differential analog to digital con- 
verter 32 where the differential value (i.e. the difference 
between the signals 25 and 27) is converted to a digital 



value. The resulting digital image signal is then passed 
to a defective-pixel filter 34 where image errors are 
detected and corrected, as described in detail below. 
[0022] The corrected digital image signal is then 
5 passed to a digital micro-interface 36 which provides an 
interface between the IC 20 and other components of 
the digital imager 2, such as the memory 12, microproc- 
essor 14 or I/O port 10. 

[0023] Also shown in Figure 2 is a digital timing 

w generator 42 which provides timing signals for operation 
of a sequential correlated doubling sampling block 44 in 
order to suppress CMOS sensor fixed pattern noise as 
taught in co-pending patent application 09/223,165, 
entitled Sequential Correlated Double Sampling Tech- 

15 niques for CMOS Area Array Sensors, which application 
is incorporated herein by reference. A row / column 
information register 46 provides information to the dig- 
ital signal controller 28 and a digital average calculator 
38 regarding where the signal currently being proc- 

20 essed originated on the sensor array (i.e. provides row 
and column information for each pixel). The imaging 
logic 8 also includes a digital signal feedback loop com- 
prising the digital average calculator 38, digital signal 
controller 28, reference control block 26, gain control 

25 block 30 and exposure time control 40. This feedback 
loop is employed to provide for optical black calibration 
and for resolution enhancement by adjusting the refer- 
ence voltage and gain for the amplifier 24. 
[0024] Further details regarding the design and 

30 operation of the defective-pixel filter 34 will now be dis- 
cussed. Referring first to Figures 3a through 3j, each 
drawing illustrates a group of three neighboring pixels in 
the image signal. The pixels are illustrated as bars, 
which bars correspond to the luminance value for the 

35 given pixel. For instance in Figure 3a, the pixel B 52 has 
a luminance value greater than that of the pixel A 50 and 
the pixel C 54 has a luminance value greater than that 
of the pixel B. The pixels A, B and C correspond to three 
adjacent pixels in the CMOS array 6. Under normal cir- 

40 cumstances, one would not expect abrupt discontinui- 
ties in the change in luminance values. For instance, as 
the image changes from dark to light, the pattern shown 
in Figure 3a, with the pixels having increasing values, 
would be expected. Likewise, in Figure 3b, the pixel B 

45 52 is darker than the pixel A 50 and the pixel C 54 is 
darker than the pixel B 52 - indicating a normal transi- 
tion from light to dark. Note that in both Figures 3a and 
3b, the luminance for the middle pixel B 52 falls in the 
range of values defined by the luminance for its neigh- 
so boring pixels A 50 and C 54. By contrast, in Figure 3c, 
the middle pixel B 52 has a luminance that is greater 
than the luminance of both of its neighbors 50 and 54, 
indicating a discontinuity in the luminance trend for the 
image. Note, however, that the pixel B 52 of Figure 3c 

55 deviates from its nearest neighbor pixel C 54 by an 
amount t. The value t indicates a threshold deviation 
between neighboring pixels that can be tolerated before 
the middle pixel will be considered as defective. Like- 
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wise, in Figures 3d through 3f, even though the lumi- 
nance of the middle pixel B 52 does not fall between the 
luminance of its neighboring pixels A 50 and C 54, the 
deviation from the nearest pixel value is equal to or less 
than t. By contrast, in Figures 3g through 3j the lumi- 
nance value for the middle pixel B 52 does not fall within 
the range of values between the neighboring pixels A 50 
and C 54, and also deviates from the value of the near- 
est pixel by more than the threshold amount t. Under the 
circumstances illustrated in Figures 3g through 3j, the 
pixel B 52 will be considered to be defective. 
[0025] In practice a row of pixels forming the image 
will be scanned with a moving three-pixel window as 
shown in Figures 4a through 4b. A pixel data stream 
preferably consists of 1 0-bit luminance values for each 
pixel of the CMOS array 6, which stream is fed to the 
defective-pixel filter 34. The defective-pixel filter 34 
applies a moving window 60 across the pixel data 
stream as it passes through the filter. In Figure 4a, the 
first three pixels 50, 52, 54 of the stream are analyzed, 
with the pixel 52 being the middle pixel B under consid- 
eration (as illustrated, pixel data is moving from right to 
left). In Figure 4b, the moving window 60 has shifted 
one pixel or, more accurately, the pixels moving through 
defective-pixel filter 34 have shifted by one pixel, and 
the pixels 52, 54, and 56 are then analyzed, with the 
pixel 54 being the middle pixel B under consideration. 
Finally, in Figure 4c, the moving window has again 
shifted by one pixel and the pixels 54, 56, and 58 are 
analyzed, with the pixel 56 being the middle pixel B 
under consideration. 

[0026] In the event that the pixel B under consider- 
ation is determined to fall outside the range of lumi- 
nance values of its neighboring pixels A and C, and to 
deviate from its nearest neighboring pixel by more than 
the threshold t, then that pixel will be flagged as defec- 
tive. In the preferred embodiments, the luminance value 
for the defective pixel will be replaced with an interpo- 
lated luminance value based upon the values of the 
neighboring pixels A and C. 

[0027] Further details regarding the defective-pixel 
filter 34 is provided with reference to Figure 5. The 
incoming pixel data is fed to a first in first out (FIFO) reg- 
ister comprising register cells 70, 72, 74, 76, and 78. 
While the preferred embodiment provides for the advan- 
tageous feature that defective pixels are identified and 
corrected for in real-time without the need for large 
memory storage, other embodiments might provide for 
a DRAM, SRAM or other type of memory in which the 
incoming pixel data is fed and stored. Note that five pix- 
els are loaded into the FIFO registers even though only 
three pixels are analyzed at one time. This is because 
the defective-pixel filter 34 can be configured for both 
monochrome images and for color images. As is known 
in the art, color image sensors interlace the pixel sen- 
sors on each row in a Bayer pattern, as is illustrated in 
Figure 6, which illustrates a portion of a color image 
sensor. In the first row 1 00 of the array, red sensors 1 02, 



104, 106 are interlaced with green sensors 103, 105, 
107. In the second row 101, green sensors 108, 110, 
112 are interlaced with blue sensors 109, 111, and 113. 
Clearly it is desirable to compare adjacent pixels of the 

5 same type (i.e. comparing red pixels to red pixels). For 
this reason, every other pixel should be selected for 
analysis, for instance pixels 1 02, 1 04, and 1 06 would be 
analyzed to determine if the pixel 104 was defective. 
Next, the pixel 1 05 would be compared to like pixels 1 03 

io and 1 07. Otherwise, if the (green) value for the pixel 1 03 
was compared to the (red) values of pixels 1 02 and 1 04, 
it would be very likely to have abrupt discontinuities, 
even though the pixel 103 was functioning normally. For 
instance, for a portion of the image in which the image 

75 was primarily red, pixels 102 and 104 would be 
expected to have high luminance values and the pixel 
103 would be expected to have very low luminance val- 
ues. 

[0028] Therefore, in the monochrome mode, the 
20 pixel A is selected from the register 72, the pixel B is 
selected from the register 74 and the pixel C is selected 
from the register 76. In the color mode, the pixel A is 
selected from register 70, the pixel B is selected from 
the register 74, and the pixel C is selected from the reg- 
25 ister 78, in order to ensure that like pixels are being 
compared. 

[0029] A multiplexer 80 selects the pixel A from 
either the register 72 or from the register 70 depending 
on whether the device is in the monochrome or color 

30 mode, and feeds the pixel value to a comparator 82. In 
the comparator 82, the value of the pixel A is compared 
to the value of the pixel B from the register 74. If the 
comparison indicates that the pixel A has a greater 
value than the pixel B, a valid logic signal (logical high) 

35 is asserted on a signal line 86, which is connected to 
one input of a two-input AND gate 88. The other input to 
the AND gate 88 is fed from the comparator 96 wherein 
the value for the pixel B is compared to the value for the 
pixel C. The pixel C is selected by a multiplexer 98 from 

40 either the register 78 or the register 76, depending upon 
whether the device is in the monochrome or color mode. 
If the pixel B is greater in value than the pixel C, then the 
comparator 96 will assert a valid signal on a signal line 
120, which is fed to the second input of the AND gate 

45 88. If both inputs to the AND gate 88 are high (indicating 
that A is greater than B and B is greater than C), the 
AND gate 88 will assert a valid signal (logical high) to a 
four-input OR gate 122. This condition corresponds to 
the situation illustrated graphically in Figure 3b, which is 

so an acceptable situation indicating that the pixel B is 
valid. The logical OR gate 122 will assert a valid signal 
(logical high) to a multiplexer 124, which will in turn 
allow the value for the pixel B, from the register 74, to be 
output from the defective -pixel filter 34 for further 

55 processing. Note that for convenience, a logical high 
signal will be treated as indicating a valid signal, 
although in other embodiments, a logical low signal 
could be used for a valid logic signal. 
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[0030] At the same time, the difference in the values 
of the pixel A and the pixel B is calculated in a block 90 
and the difference is compared to the threshold t in a 
comparator 92. Note that the value for the threshold t 
can be selected by a user and stored in a register 94. 5 
Alternatively, or if no value for t is selected by the user, 
a default value for t can be stored in the register 94. In 
some embodiments, the threshold value can be auto- 
matically generated based upon an iterative process in 
which a feedback signal indicative of image quality is 
compared to a varying value for t until a threshold value 
is reached that effectively cancels out defective pixels 
without canceling out desired luminance variations that 
occur naturally in the image. 

[0031] If the difference between the pixels A and B 
is less than or equal to the threshold, this also indicates 
that the pixel B is valid, corresponding to the condition in 
Figures 3d or 3f. Note that it does not matter whether 
the pixel B is greater than or less than the pixel A, as 
long as the difference is less than or equal to the thresh- 
old. If so, then the comparator 92 will output a logical 
high to the OR gate 122, which will also cause the mul- 
tiplexer 124 to allow the pixel B to be passed. 
[0032] Likewise, if the difference between the pixels 
B and C is less than or equal to the threshold value, cor- 
responding to Figure 3c or 3e, than the pixel B is valid 
(regardless of the difference in value between the pixels 
A and B). This condition is determined by a block 126 
and a comparator 1 28 and if the difference between pix- 
els the B and C is less than t, a logical high is asserted 
to an OR gate 122 and the value for the pixel B is 
passed to the output from the register 74 via a multi- 
plexer 124. 

[0033] The fourth input of the OR gate 122 is fed 
from an AND gate 130. The first input to the AND gate 
130 is the inverted output of the comparator 82 (via an 
inverter 1 32) and the second input is the inverted output 
of the comparator 96 (via an inverter 134). The AND 
gate 130 will output a valid signal (logical high) to the 
OR gate 122 when the pixel C is greater in value than 
the pixel B and the pixel B is greater in value than the 
pixel A. This corresponds to Figure 3a. 
[0034] If none of the above conditions is met, the 
inputs to the OR gate 122 will be logical lows, and 
hence the control input to the multiplexer 124 will be a 
logical low. This indicates that the pixel B is defective 
(corresponding to one of the conditions of Figures 3g 
through 3j). Under those circumstances, the multiplexer 
124 will pass a corrected pixel B value to the output. 
This corrected pixel value is calculated in a block 136. In 
the preferred embodiments, the corrected pixel B value 
is calculated from the average value of its neighboring 
pixels A and C, in other words = (A + C) / 2 . 

The advantage of using a simple interpolation between 
the pixels A and C is that those values are already 
stored in the FIFO register. More complex interpolations 
could be employed to generate the value for B corrected , 
such as using the values of the nearest two or three 



neighbors on either side of B, but such interpolations 
would require additional storage elements in which the 
neighboring pixel values are stored and would also 
require additional combinational logic (with its associ- 
ated costs and real estate requirements). 
[0035] In the preferred embodiments, the circuit of 
Figure 5 is realized in combinational logic on a semicon- 
ductor chip using CMOS fabrication processes. Advan- 
tageously, the preferred embodiment filter is fabricated 
on the same chip as the sensor array, as provided for 
with a CMOS sensor array. While other types of arrays 
such as CCD array may also be used, they may not be 
as desirable for reasons of processing differences. In 
other embodiments, the function could be achieved by a 
microprocessor running a sequence of program instruc- 
tions. Alternatively, the circuitry could be realized in pro- 
grammable gate array logic or other programmable 
logic. 

[0036] As will be apparent from the above descrip- 
tion, the preferred embodiments provide several advan- 
tageous features including the ability to eliminate 
temperature and time dependent pixel defects and the 
ability to filter both white pixels and dark pixels with 
adjustable thresholds. The adjustable threshold feature 
allows compensation for image spatial frequency and 
for degree of array pixel defects. Additionally, the pre- 
ferred embodiments operate at high speed in real time 
and do not require a frame or line memory to store an 
entire frame or line of data. Both the logic and the sen- 
sor array can be formed on a single chip and are com- 
patible with CMOS operations. The method can be 
employed as a part of a wafer probe operation in order 
to determine wafer yield and is operable in either color 
or monochrome image modes. The described embodi- 
ments maintain high frequency edge components, such 
as a rapid transition from a dark to very bright object, or 
from a bright to dark object. 

[0037] While this invention has been described with 
reference to illustrative embodiments, this description is 
not intended to be construed in a limiting sense. Various 
modifications and combinations of the illustrative 
embodiments, as well as other embodiments of the 
invention, will be apparent to persons skilled in the art 
upon reference to the description. It is therefore 
intended that the appended claims encompass any 
such modifications or embodiments. 

Claims 

1 . A filter for filtering an image which includes a plural- 
ity of picture elements each represented by a lumi- 
nance or a chrominance value, including: 

means for selecting a picture element and its 
two adjacent picture elements, 
means for determining whether the selected 
picture element lies between the two adjacent 
picture elements in value, 
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means for determining whether the selected 
picture element differs in value from both of the 
two adjacent picture elements by more than a 
set amount and 

means for replacing the selected picture ele- 5 
ment, in value, by a value dependent on the 
two adjacent picture elements, when, in value, 
the selected picture element does not lie 
between the two adjacent picture elements and 
differs from both of the two adjacent picture 10 
elements by more than the set amount and 
means for repeating the procedure for further 
picture elements serving as the selected pic- 
ture element. 

75 

2. A filter as claimed in claim 1 , including means for 
replacing the selected picture element, in value, by 
an average of the values of the two adjacent picture 
elements. 

20 

3. A filter as claimed in claim 1 , including means for 
replacing the selected picture element, In value, by 
an interpolation dependent on the values of the two 
adjacent picture elements. 

25 

4. A filter as claimed in any one of claims 1 to 3 com- 
prising: a first comparator circuit for receiving as 
input a first picture element value and a second pic- 
ture element value and outputting a valid logic sig- 
nal to a first input of an AND circuit and to a first 30 
inverter when the first picture element value is 
greater than the second picture element value, 

a first difference calculator for receiving as 
input said first picture element value and said 35 
second picture element value and outputting to 
a first input of a second comparator circuit a 
first difference value corresponding to the dif- 
ference between said first and second picture 
element values; 40 
said second comparator circuit also for receiv- 
ing as input a threshold value and outputting a 
valid logic signal to a first input of an OR circuit 
when the first difference value is less than or 
equal to the set amount; 45 
a third comparator circuit for receiving as input 
said second picture element value and a third 
picture element value and outputting a valid 
logic signal to a second input of said first AND 
circuit and to a second inverter when the sec- so 
ond picture element value is greater than the 
third picture element value and 
a second difference calculator for receiving as 
input said second picture element value and 
said third picture element value and outputting 55 
a second difference value corresponding to the 
difference between said second and third pic- 
ture element values to a first input of a fourth 



comparator circuit, 

said fourth comparator circuit also for receiving 
as input the set amount threshold value and 
outputting a valid logic signal to a third input of 
said OR circuit when the second difference 
value is less than or equal to the set amount 
and 

a second AND circuit coupled to said first and 
second inverters and having an output coupled 
to a fourth input of said OR circuit. 

5. A filter as claimed in claim 4 further comprising: 

a corrected picture element calculation block 
for receiving as input said first and third picture 
element values and outputting a corrected pic- 
ture element value, 

a multiplexer having a first input coupled to the 
output of said corrected picture element calcu- 
lation block and receiving as a second input 
said picture element value and having a control 
input coupled to an output of said OR circuit. 

6. A filter as claimed in any one of claims 1 to 5, fur- 
ther comprising registers for storing picture element 
values. 

7. A filter as claimed in any one of claims 4 to 6, capa- 
ble of operating in both monochrome and color 
mode and further comprising: 

at least one multiplexer for receiving as input 
the picture element values stored in at least 
two registers and outputting one of said picture 
element values in response to an indication of 
monochrome or color mode operation. 

8. A filter as claimed in any one of claims 4 to 7, 
wherein said first, second, and third picture element 
values are ten bit digital values. 

9. A filter as claimed in any one of claims 1 to 8, fur- 
ther comprising a threshold value register for stor- 
ing the said set amount. 

10. A filter as claimed in claim 9, including means per- 
mitting a user to enter the set amount in the thresh- 
old value register. 

11. A filter as claimed in claim 9, including means for 
providing the set amount from a feedback signal 
indicative of image quality. 

12. A filter as claimed in any one of claims 4 to 8, 
wherein said comparator circuits, difference calcu- 
lators, and registers are fabricated with CMOS 
processes. 
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13. A filter as claimed in any one of claims 1 to 12, 
including a general purpose microprocessor. 

14. A digital imaging device including a filter as claimed 
in any one of claims 1 to 13 comprising: 

a substrate, 

a sensor array formed on the substrate, the 
array generating an electrical signal corre- 
sponding to the amount of light impinging upon 
the array and 

imaging logic formed on the substrate, coupled 
to the sensor array and receiving the electrical 
signal, the imaging logic including an analog to 
digital converter receiving the electrical signal 
and outputting digital picture element values. 

15. A digital imaging device as claimed in claim 14, fur- 
ther comprising a first in first out register wherein 
said picture element values are stored. 

16. A digital imaging device as claimed in claim 14 or 
claim 15, wherein the filter operates in real time. 

17. A digital imaging device as claimed in any one of 
claim 14 to 16, wherein the sensor array is com- 
prised of CMOS cells. 

18. A digital imager including a filter as claimed in any 
one of claims 1 to 13 comprising: 

a lens mechanism, 

a sensor array positioned within a focal plane of 
said lens mechanism, 

an analog buffer and amplifier coupled to an 
output of said sensor array and 
imaging logic coupled to said amplifier, the filter 
being included in the said imaging logic. 

19. A digital imager as claimed in claim 18, wherein 
said sensor array and said imaging logic are fabri- 
cated on a single integrated circuit. 

20. A digital imager as claimed in claim 18 or 19, 
wherein the sensor array is comprised of CMOS 
cells. 

21. A digital imager as claimed in claim 18 or 19, 
wherein the sensor array is comprised of CCD 
cells. 

22. A digital imager as claimed in any one of claims 18 
to 21 , wherein the imaging logic includes combina- 
tional logic. 

23. A digital imager as claimed in any one of claims 18 
to 22, wherein the imaging logic includes a micro- 
processor. 



24. A method of filtering an image including a plurality 
of picture elements each represented by a lumi- 
nance or a chrominance value, including the steps 
of: 

5 

selecting a picture element and its two adjacent 
picture elements, 

determining whether the selected picture ele- 
ment lies, in value, between the two adjacent 
10 picture elements, 

determining whether the selected picture ele- 
ment differs, in value, from both of the two adja- 
cent picture elements by more than a set 
amount and 

is when the selected picture element does not lie, 

in value, between the two adjacent picture ele- 
ments and differs from both of the two adjacent 
picture elements by more than the set amount, 
replacing the selected picture element, in 

20 value, by a value dependent on the two adja- 

cent picture elements, 

the procedure being repeated for further pic- 
ture elements serving as the selected picture 
element 

25 

25. A method as claimed in claim 24 comprising: 

determining a first difference value between 
the luminance value of the selected picture ele- 
30 ment and the luminance value of a first neigh- 

boring picture element, 

comparing the first difference value to the set 
value, 

determining a second difference value 
35 between the luminance of the selected picture 

element and the luminance value of a second 
neighboring picture element, 
comparing the second difference value to the 
set value, 

40 detecting whether the luminance value for the 

selected picture element falls within an accept- 
able range defined by the luminance value for 
the first neighboring picture element and the 
luminance value for the second neighboring 

45 picture element and 

identifying the selected picture element as 
defective if the luminance value for the selected 
picture element does not fall within the accept- 
able range and neither the first difference value 

50 nor the second difference value is less than or 

equal to the set value. 

26. The method as claimed in claim 24 or claim 25, 
wherein an average of the luminance value of the 

55 first neighboring picture element and the luminance 
value of the second neighboring picture element 
replaces the selected picture element in value. 
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27. A method as claimed in claim 24 or claim 25, 
wherein an interpolated value taking account of two 
or more neighboring picture elements on either side 
of the selected picture element replaces the 
selected picture element in value. 5 

28. A method as claimed in any one of claims 24 to 27, 
wherein the set value is entered in a register by a 
user. 

10 

29. A method as claimed in any one of claims 24 to 27, 
wherein the set value is determined from a feed- 
back signal indicative of image quality. 

30. A method as claimed in any one of claims 24 to 29, 15 
performed in real time. 
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